Exclude /registry/ placeholder from www sitemap (fixes cross-sitemap lastmod duplicate) - #21139

What

content/registry.md is excluded from the www sitemap via a new sitemap_exclude front-matter flag, wired into layouts/_default/sitemap.xml. Documented in CONTRIBUTING.md.

Why

https://www.pulumi.com/registry/ is the only URL on pulumi.com that shows up in two different submitted sitemaps at once, with lastmod values 861+ days apart:

The 2023-12-11 value is also the single oldest lastmod in the entire 2,738-URL www sitemap, next oldest being 831 days.

content/registry.md is not a real content page. Its own front matter says so: it exists only so S3 has an object to hand to CloudFront at registry/index.html, so CloudFront can then forward the actual request on to the Registry app, which is what really serves /registry/. Google never sees the HTML Hugo renders here. Confirmed the git history is genuinely frozen at that 2023 commit (2c85e02c, "Replace infinite redirect with placeholder page (#10300)") via an unshallowed clone, so this is not a rollup-fix gap like the one PR #21036 closed for /blog/ -- this page has no descendants to roll up from, and its content is not the descendant's content either.

The follow-up card that flagged this asked whether to fix it with an explicit lastmod front-matter override refreshed by a script, or accept the git date as inherently uninformative. Neither is right: a script-maintained date on a deliberately empty file still leaves two sitemaps disagreeing about one canonical URL, and accepting the status quo leaves the property's only cross-sitemap duplicate and its single oldest lastmod in place. The Registry app's sitemap already carries this URL with an accurate value, so the correct fix is to stop claiming it a second time here.

Why not reuse private or block_external_search_index

Both of those existing front-matter flags also add a noindex meta tag (see layouts/partials/head.html). /registry/ is correctly indexed today (Search Console: Submitted and indexed, crawled within the last day) and should stay that way -- only the sitemap entry is wrong, not the page's indexability. sitemap_exclude is a new, narrowly-scoped flag that touches only layouts/_default/sitemap.xml's inclusion check.

Impact, honestly sized

This is a lastmod-trust fix, not a traffic play. Search Console already shows Google crawling and ranking /registry/ and the wider registry section fine (90d: landing page 382 clicks / 9.8K impressions / avg pos 13.0; the whole /registry/* section 29.9K clicks / 4.95M impressions / avg pos 7.2, driven by the long tail of individual provider pages). Google's own guidance is that it uses <lastmod> "if it's consistently and verifiably... accurate" (Build and Submit a Sitemap); a URL that contradicts itself across two sitemaps on the same property is exactly the kind of signal that erodes that trust property-wide, including across the 59,633 URLs in the registry sitemap where lastmod-driven crawl scheduling is doing real work.

Verification

Full local hugo --gc --minify -e production build, before and after this change, diffing the generated sitemap.xml URL sets:

$ diff sitemap_before.txt sitemap_after.txt
1973d1972
< https://www.pulumi.com/registry/

Exactly one URL removed, nothing else changed (2,162 -> 2,161 URLs). public/registry/index.html still renders (92,876 bytes) after the change, so the CloudFront placeholder trick this file exists for is unaffected.

Out of scope (follow-up cards filed on the Marketing Content Calendar board)

• The Registry app's own landing-page lastmod (2026-04-22) is itself git-date-based on a rarely-touched file and doesn't track the page's actual live content -- lower priority since it's not wrong, just not a great freshness signal.
• A duplicated <title>Pulumi Registry | Pulumi Registry</title> tag on the live registry landing page, traced to themes/default/layouts/partials/head.html in the pulumi/registry repo.
• The nested sitemap index at /registry/sitemap.xml (a <sitemapindex> pointed to by /sitemap-index.xml, itself pointing to two more sitemaps) -- Google has stated (via John Mueller) that nested sitemap indexes are not supported.

🧠 This PR was created by workprentice.